Subject tracking device and camera

ABSTRACT

A subject tracking device includes: a first similarity factor calculation unit that compares an input image assuming characteristics quantities corresponding to a plurality of characteristics components, with a template image assuming characteristics quantities corresponding to the plurality of characteristics components, and calculates a similarity factor indicating a level of similarity between the input image and the template image in correspondence to each of the plurality of characteristics components; a normalization unit that normalizes similarity factors corresponding to the plurality of characteristics components having been calculated by the first similarity factor calculation unit; and a second similarity factor calculation unit that calculates a similarity factor indicating a level of similarity between the input image and the template image based upon results of normalization achieved via the normalization unit.

INCORPORATION BY REFERENCE

The disclosure of the following priority application is hereinincorporated by reference:

Japanese Patent Application No. 2009-093676 filed Apr. 8, 2009

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a subject tracking device and a camera.

2. Description of Related Art

Japanese Patent Publication No. 3768073 discloses an object trackingdevice. The object tracking device disclosed in the publication tracks aphotographic subject by calculating the level of similarity manifestingbetween a template and an image through an arithmetic operation methodknown as normalized correlation.

There is an issue that the similarity level calculation methods normallyadopted in object tracking devices in the related art such as thatmentioned above, fail to address effectively in that a high level ofaccuracy cannot be assured for the similarity level calculated for animage with different gains applied in correspondence to various imagecomponents such as the brightness and the chrominance.

SUMMARY OF THE INVENTION

A subject tracking device according to a first aspect of the presentinvention comprises: a first similarity factor calculation unit thatcompares an input image assuming characteristics quantitiescorresponding to a plurality of characteristics components, with atemplate image assuming characteristics quantities corresponding to theplurality of characteristics components, and calculates a similarityfactor indicating a level of similarity between the input image and thetemplate image in correspondence to each of the plurality ofcharacteristics components; a normalization unit that normalizessimilarity factors corresponding to the plurality of characteristicscomponents having been calculated by the first similarity factorcalculation unit; and a second similarity factor calculation unit thatcalculates a similarity factor indicating a level of similarity betweenthe input image and the template image based upon results ofnormalization achieved via the normalization unit.

According to a second aspect of the present invention, in the subjecttracking device according to the first aspect, it is preferable that thenormalization unit normalizes the similarity factors corresponding tothe plurality of characteristics components by multiplying thesimilarity factors by normalizing values used to equalize the similarityfactors corresponding to the individual characteristic components.

According to a third aspect of the present invention, the subjecttracking device according to the second aspect may further comprise aweighting unit that weights the similarity factors, having beencalculated for the plurality of characteristics components, incorrespondence to characteristics of the input image.

According to a fourth aspect of the present invention, in the subjecttracking device according to the second aspect, it is preferable thatthe normalizing values are calculated and recorded into a storage mediumin advance; and the normalization unit reads out the normalizing valuesfrom the storage medium and executes normalization.

According to a fifth aspect of the present invention, in the subjecttracking device according to the first aspect, it is preferable that theplurality of characteristics components are a brightness component and achrominance component of the image; and the first similarity factorcalculation unit calculates a similarity factor indicating a level ofsimilarity between the brightness component of the input image and thebrightness component of the template image and a similarity factorindicating a level of similarity between the chrominance component inthe input image and the chrominance component in the template image.

A camera according to a sixth aspect of the present invention isequipped with a subject tracking device according to the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the structure of a camera achieved inan embodiment of the present invention;

FIG. 2 presents a flowchart of the template matching processing;

FIGS. 3A and 3B present a schematic illustration of a specific exampleof a similarity factor calculated without normalization;

FIGS. 4A and 4B present a schematic illustration of a specific exampleof a similarity factor calculated by using normalized values; and

FIGS. 5A to 5D present a schematic illustration of a specific example ofa similarity factor calculated by weighting the normalized values.

DESCRIPTION OF PREFERRED EMBODIMENTS First Embodiment

FIG. 1 is a block diagram showing the structure adopted in a cameraequipped with a subject tracking device achieved in the first embodimentof the present invention. The camera 100 comprises an operation member101, a lens 102, an image sensor 103, a control device 104, a memorycard slot 105 and a monitor 106. The operation member 101 includesvarious input members operated by the user, such as a power button, ashutter release button, a zoom button, a cross key, an OK button, areproduce button and a delete button.

While the lens 102 is constituted with a plurality of optical lenses,the plurality of optical lenses are represented by a single lens inFIG. 1. The lenses constituting the lens 102 include a zoom lens used toadjust the zoom magnification rate and a focus adjustment lens (AF lens)used to adjust the focusing condition. The image sensor 103, which maybe, for instance, a CCD image sensor or a CMOS image sensor, captures asubject image formed through the lens 102. The image sensor 103 thenoutputs image signals expressing the image having been captured to thecontrol device 104.

Based upon the image signals input thereto from the image sensor 103,the control device 104 generates image data (hereafter referred to as“main image data”) in a predetermined image format such as the JPEGformat. In addition, based upon the main image data thus created, thecontrol device 104 generates display image data such as thumbnail imagedata. The control device 104 then creates an image file that containsthe main image data and the thumbnail image data having been generatedas well as additional header information, and outputs the image file tothe memory card slot 105.

The image file output from the control device 104 is written and thusrecorded into a memory card that is loaded at the memory card slot 105and is utilized as a storage medium. In addition, in response to aninstruction issued by the control device 104, an image file, storedwithin a memory card loaded into the memory card slot 105, is read out.

The monitor 106 is a liquid crystal monitor (rear side monitor) mountedat the rear surface of the camera 100. At the monitor 106, an imagestored in the memory card, a setting menu in which settings for thecamera 100 are selected, or the like is brought up on display. Inaddition, as the user sets the camera 100 in a photographing mode, thecontrol device 104 outputs to the monitor 106 display image data forimages obtained in time series from the image sensor 103. As a result, athrough image or live view image is displayed at the monitor 106.

The control device 104, constituted with a CPU, a memory and otherperipheral circuits, controls the camera 100. It is to be noted that thememory constituting the control device 104 includes an SDRAM and a flashmemory. The SDRAM, which is a volatile memory, is used by the CPU as awork memory where a program to be executed is opened or as a buffermemory where data are temporarily recorded. In the flash memory, whichis a non-volatile memory, program data related to the program executedby the control device 104, various parameters that are read duringprogram execution and the like are recorded.

The control device 104 in the embodiment executes template matchingprocessing for each frame of live view image input from the image sensor103 in reference to a template image prepared in advance and identifiesan image area within the frame manifesting similarity to the templateimage as a subject area. The control device 104 then executes subjecttracking processing through frame-to-frame tracking of the identifiedsubject area.

The following is a detailed description of the template matchingprocessing executed in the embodiment, given in reference to theflowchart presented in FIG. 2. It is to be noted that the processingshown in FIG. 2 is executed by the control device 104 based upon aprogram that is started up as a live view image photographing operationstarts.

In step S1, the control device 104 designates the image having beeninput (hereafter referred to as the “input image”) as a matching target,and then the operation proceeds to step S2. In step S2, the controldevice 104 slices out an image portion within the input image, whichranges over an area matching the size of the template image, anddesignates the image portion thus sliced out as a comparison targetimage. The operation then proceeds to step S3 in which the controldevice 104 compares the comparison target image having been sliced outwith the template image and calculates a similarity factor. It is to benoted that the method adopted in the embodiment when calculating thesimilarity factor is to be described later. The operation then proceedsto step S4.

In step S4, the control device 104 makes a decision as to whether or notthe similarity factor has been calculated in correspondence to eachcomparison target image having been sliced out from a search area, e.g.,from the entire range of the input image or from a specific area setwithin the input image. If a negative decision is made in step S4, theoperation returns to step S2 to repeat the processing. It is to be notedthat the control device 104 repeats the processing by offsetting theextraction position at which it slices out the comparison target area instep S2 from the previous extraction position and thus, similarityfactors are ultimately calculated to indicate the similarity between thetemplate image and the comparison target images sliced out from theentire search area. If an affirmative decision is made in step S4, theoperation proceeds to step S41.

In step S41, the control device 104 determines the extraction positionwithin the input image at which the comparison target image achievingthe highest level of similarity to the template image has been slicedout, based upon the similarity factors calculated in step S3 anddesignates the extraction position thus determined as a subjectposition. It is to be noted that the similarity factor calculated for acomparison target image through the similarity factor calculation to bedetailed later assumes a smaller value (calculated similarity factorvalue) when the similarity of the comparison target image to thetemplate image is higher. Accordingly, the control device 104 designatesthe extraction position at which the comparison target image with thesmallest similarity factor value has been sliced out as the subjectposition in step S41. The operation then proceeds to step S5.

In step S5, the control device 104 makes a decision as to whether or notthe smallest similarity factor value calculated in correspondence to thesubject position having been designated in step S41 (the similarityfactor value having been calculated at the extraction position at whichthe comparison target image with the highest level of similarity hasbeen sliced out) is equal to or less than a predetermined thresholdvalue (hereafter referred to as an “update threshold value”). If anegative decision is made in step S5, the operation proceeds to step S7,which is to be detailed later. If, on the other hand, an affirmativedecision is made in step S5, the operation proceeds to step S6.

In step S6, the control device 104 updates the template image bydesignating the comparison target image for which the smallestsimilarity factor value has been calculated as the new template image,and then the operation proceeds to step S7. While the shape of thesubject may continuously change, the subject can still be reliablytracked without ever losing it by sequentially updating the templateimage with the sliced-out image achieving a high level of similarity tothe current template image as described above.

In step S7, the control device 104 makes a decision as to whether or notall the frames have been processed, i.e., whether or not the live viewimage input has stopped. If a negative decision is made in step S7, theoperation returns to step S1 to repeat the processing. However, if anaffirmative decision is made in step S7, the processing ends.

Next, a method that may be adopted in the embodiment to calculate asimilarity factor indicating the level of similarity between thetemplate image and a comparison target image is described. The followingdescription is provided by assuming that the similarity factor iscalculated through the SAD (sum of absolute differences) method, whichis one of the similarity factor calculation methods in the related art.It is to be noted that in the SAD method, the sums of absolutedifferences are each calculated through a pixel-by-pixel comparison ofthe comparison target image and the template image and then a similarityfactor is calculated by adding up the sums.

First, a typical SAD-based similarity factor calculation method isdescribed. Assuming that the template image and the comparison targetimage are both expressed in the YCbCr colorimetric system constitutedwith three components; the brightness component Y and the chrominancecomponents Cb and Cr, similarity factors each corresponding to one ofthe three components can be calculated as expressed in (1) to (3) below.It is to be noted that SAD_(Y), SAD_(Cr) and SAD_(Cb) in the followingexpressions (1) to (3) respectively represent the Y component similarityfactor, the Cr component similarity factor and the Cb componentsimilarity factor. In addition, the three components in the comparisontarget image are respectively notated as Y_(image), Cr_(image) andCb_(image), whereas the three components in the template image arerespectively notated as Y_(template), Cr_(template) and Cb_(template).SAD_(Y) =Σ|Y _(image) −Y _(template)|  (1)SAD_(Cr) =Σ|Cr _(image) −Cr _(template)|  (2)SAD_(Cb) =Σ|Cb _(image) −Cb _(template)|  (3)

As indicated in expression (4) below, the similarity factor (SAD)indicating the level of similarity between the template image and thecomparison target image is normally calculated by adding up thesimilarity factors corresponding to the individual components havingbeen calculated as expressed in (1) to (3).SAD=SAD_(Y)+SAD_(Cr)+SAD_(Cb)  (4)

However, the similarity factor calculated as expressed in (1) to (4)above is bound to contain the individual component similarity factors towhich gains reflecting the pixel information and assuming valuesdifferent from one another, are applied. For instance, provided thateach pixel holds information corresponding to the individual componentsY, Cb and Cr, SAD_(Y) calculated as expressed in (1) is bound to assumea value 4 to 6 times larger than the value calculated for SAD_(Cr) asexpressed in (2) or the value calculated for SAD_(Cb) as expressed in(3). As expression (4) indicates, the similarity factor SAD iscalculated by adding up SAD_(Y), SAD_(Cr) and SAD_(Cb), and thus, if thegain applied to even one of the components is different, the similarityfactor SAD will be calculated without uniformly evaluating theindividual components.

For instance, the gain applied to the Y component similarity factor(SAD_(Y)) may be greater than the gains applied to the SAD_(Cr) andSAD_(Cb), as shown in FIG. 3A. Under such circumstances, the similarityfactor SAD calculated as expressed in (4), which is greatly affected bythe value of SAD_(Y), may take on the smallest value at an extractionposition 3 b, i.e., a non-subject position, at which SAD_(Y) assumes thesmallest value, even though SAD_(Cr) and SAD_(Cb) both assume thesmallest values at another extraction position 3 a, which is actuallythe accurate subject position. In such a case, the extraction position 3b will be erroneously designated as the subject position incorrespondence to the particular frame.

In order to address this problem, the control device 104 in theembodiment calculates a similarity factor SAD only after normalizing theindividual component similarity factors (SAD_(Y), SAD_(Cr), SAD_(Cb)).In order to enable normalization of the individual component similarityfactors, similarity factor normalizing values, e.g., a similarity factornormalizing value N_(y) used to normalize SAD_(Y), a similarity factornormalizing value N_(Cr), used to normalize SAD_(Cr) and a similarityfactor normalizing value N_(Cb) used to normalize SAD_(Cb), are recordedin advance in the flash memory in the camera 100.

A method that may be adopted when calculating the similarity factornormalizing values N_(y), N_(Cr) and N_(Cb) in correspondence to thevarious components is now described. First, similarity factor averagesare calculated in correspondence to the individual components asexpressed in (5) to (7) below by using dynamic image data expressingdynamic images captured in standard subject tracking scenes.

$\begin{matrix}{\overset{\_}{{SAD}_{y}} = {\frac{1}{n}{\sum{SAD}_{Y}}}} & (5) \\{\overset{\_}{{SAD}_{Cr}} = {\frac{1}{n}{\sum{SAD}_{Cr}}}} & (6) \\{\overset{\_}{{SAD}_{Cb}} = {\frac{1}{n}{\sum{SAD}_{Cb}}}} & (7)\end{matrix}$

It is to be noted that the term “standard scenes” is used to indicatescenes selected by excluding non-standard scenes such as a scenecaptured through monochromatic photographic operation. In addition,expressions (5) to (7) indicate that the similarity factor averages arecalculated by using dynamic image samples collected in n differentscenes.

Then, the similarity factor normalizing values N_(y), N_(Cr), andN_(Cb)) are calculated as expressed in (8) to (10) by taking thereciprocals of the individual component similarity factor averageshaving been calculated as expressed in (5) to (7). The embodiment isdescribed by assuming that the similarity factor normalizing valuesN_(y), N_(Cr) and N_(Cb) calculated as expressed in (8) to (10) arerecorded in the flash memory of the camera 100 in advance.

$\begin{matrix}{N_{y} = \frac{1}{\overset{\_}{{SAD}_{y}}}} & (8) \\{N_{Cr} = \frac{1}{\overset{\_}{{SAD}_{Cr}}}} & (9) \\{N_{Cb} = \frac{1}{\overset{\_}{{SAD}_{Cb}}}} & (10)\end{matrix}$

The control device 104 in the embodiment reads out the similarity factornormalizing values N_(y), N_(Cr) and N_(Cb) recorded in the flash memoryand calculates the similarity factor SAD indicating the level ofsimilarity between the template image and the comparison target image byfirst multiplying the individual component similarity factors, i.e.,SAD_(Y), SAD_(Cr) and SAD_(Cb) respectively by the similarity factornormalizing values N_(y), N_(Cr) and N_(Cb) having been read out andthus normalizing the individual component similarity factors.

In more specific terms, the control device 104 is able to calculate thesimilarity factor SAD indicating the level of similarity between theframe and the template image by first normalizing the individualcomponent similarity factors, as expressed in (11) below. It is to benoted that in the following expression (11), the entire sum is dividedby 3, since the SAD as a whole is constituted with three differentcomponents, i.e., SAD_(Y), SAD_(Cr) and SAD_(Cb).

$\begin{matrix}{{SAD} = {\frac{1}{3}\left( {{N_{y} \cdot {SAD}_{Y}} + {N_{Cr} \cdot {SAD}_{Cr}} + {N_{Cb} \cdot {SAD}_{Cb}}} \right)}} & (11)\end{matrix}$

Since SAD is calculated only after the SAD values corresponding to theindividual components are normalized as indicated in expression (11), auniform weight is applied to the various components and the variouscomponent similarity factors (the SAD values corresponding to thevarious components) are output with a uniform gain applied thereto. As aresult, even when a significantly larger gain is initially applied to agiven component similarity factor, the similarity factor correspondingto the particular component is not allowed to cause the wrong positionwithin the frame to be erroneously designated as the subject position.

For instance, even when the gain applied to SAD_(Y) is greater than thegains applied to SAD_(Cr) and SAD_(Cb) as shown in FIG. 3A, the gainscorresponding to the various similarity factors can be equalized throughnormalization as shown in FIG. 4A. Thus, the sum of the normalizedsimilarity factors corresponding to the various components takes on thesmallest value at an extraction position 4 a, which is the correctsubject position, as indicated in FIG. 4B, and consequently, the subjectposition can be determined accurately.

The control device 104 in the embodiment simply needs to calculate thesimilarity factor indicating the level of similarity between thetemplate image and each comparison target image as expressed in (11) instep S3 in FIG. 2. Then, after calculating the similarity factors forthe entire input image (after making an affirmative decision in stepS4), the control device is able to determine the subject position withinthe input image simply by designating the extraction position at whichthe comparison target image with the least SAD has been sliced out asthe subject position. The control device 104, repeatedly executing theprocessing described above through a plurality of frames, is able totrack the subject by identifying the subject position within each frame.

The following operational effects are achieved through the firstembodiment described above.

(1) The control device 104 compares a frame of live view image with thetemplate image for the brightness component (Y component) and thechrominance components (Cr component and Cb component), which assumespecific values to characterize each image and calculates the similarityfactors SAD_(Y), SAD_(Cr) and SAD_(Cb) in correspondence to theindividual components. The control device 104 then normalizes theindividual component similarity factors by multiplying the calculatedcomponent similarity factors by similarity factor normalizing values.Based upon the normalized similarity factors corresponding to theindividual components, the control device 104 calculates the similarityfactor SAD indicating the level of similarity between the frame and thetemplate image. As a result, the level of subject tracking performanceis improved through the normalization of the individual componentsimilarity factors.(2) Similarity factor normalizing values to be used to equalize theindividual component similarity factor values are calculated in advanceand the control device 104 normalizes the individual componentsimilarity factors so as to equalize the individual component similarityfactor values by using the similarity factor normalizing values. As aresult, even when a significantly larger gain is initially applied to agiven component similarity factor, the similarity factor correspondingto the particular component is not allowed to cause the wrong positionwithin the frame to be erroneously designated as the subject position.

Second Embodiment

In the first embodiment described above, the individual componentsimilarity factors, i.e., SAD_(Y), SAD_(Cr) and SAD_(Cb) are firstnormalized by multiplying them by the similarity factor normalizingvalues N_(y), N_(Cr) and N_(Cb) and then the similarity factor SADindicating the level of similarity between the frame and the templateimage is calculated based upon the normalized individual componentsimilarity factors, so as to improve the level of subject trackingperformance.

In the second embodiment, after normalizing the individual componentsimilarity factors as in the first embodiment, the similarity factor SADis calculated by weighting the normalized brightness componentsimilarity factor (SAD_(Y)) or the normalized chrominance is componentsimilarity factors (SAD_(Cr) and SAD_(Cb)), so as to further improve thelevel of the subject tracking performance. It is to be noted that anyaspect of the second embodiment to which FIGS. 1 to 3 also apply, thatis similar to the first embodiment is not repeatedly explained and thatthe following explanation focuses on the features distinguishing it fromthe first embodiment.

The control device 104 in the second embodiment weights the normalizedcomponents based upon the level of color saturation in thematching-target frame. For instance, if the level of color saturation inthe matching-target frame is high, the tracking performance can beimproved with the similarity factor calculated by targeting thechrominance components for evaluation rather than the brightnesscomponent and accordingly, SAD is calculated by applying greater weightto the normalized chrominance component similarity factors (SAD_(Cb) andSAD_(Cr)). If, on the other hand, the color saturation in thematching-target frame is low, better tracking performance can be assuredwith the similarity factor calculated by targeting the brightnesscomponent for evaluation rather than the chrominance components andaccordingly, SAD is calculated by applying greater weight to thenormalized brightness component similarity factor (SAD_(Y)).

For instance, the control device 104 may calculate SAD as expressed in(12) below by using a parameter α in weighting the normalized similarityfactors corresponding to the various components.

$\begin{matrix}{{SAD} = {\frac{1}{3}\left( {{2 \cdot \left( {1 - \alpha} \right) \cdot N_{y} \cdot {SAD}_{Y}} + {\alpha \cdot \begin{pmatrix}{{N_{Cr} \cdot {SAD}_{Cr}} +} \\{N_{Cb} \cdot {SAD}_{Cb}}\end{pmatrix}}} \right)}} & (12)\end{matrix}$

It is to be noted that the parameter α in the embodiment, whichcorresponds to a parameter indicating the color saturation and assuminga range of 0<α<1, takes on the form of a function expressed asα=1/(1+e^(−ax+b)) provided that x=(1/number of pixels)·Σ(|Cr|+|Cb|).

By weighting the individual component similarity factors, which havebeen normalized, based upon the parameter α as described above, SAD canbe calculated based upon the individual component similarity factorsweighted in correspondence to the saturation level of the image afteradjusting the gains applied to the similarity factors for uniformityand, as a result, a further improvement in the level of subject trackingperformance is achieved.

When the color saturation of the matching-target frame is high, i.e.,when the matching-target frame is highly chromatic, the parameter ashould assume a larger value, so as to calculate the similarity factorby heavily weighting the chrominance components for evaluation over thebrightness component and ultimately improve the tracking performance.

When the matching-target frame is highly chromatic, a may be set to, forinstance, 0.8. In such a case, the brightness component SAD_(Y) will bemultiplied by 0.2×2 and the chrominance components SAD_(Cr) and SAD_(Cb)will each be multiplied by 0.8 in expression (12) as indicated in FIG.5A. As a result, the similarity factor SAD can be calculated by loweringthe extent to which the brightness component SAD_(Y) factors into thecalculation. The similarity factor SAD thus calculated, less affected bySAD_(Y), takes on the smallest value at an extraction position 5 a,i.e., the correct subject position, as shown in FIG. 5B, to allow thesubject position to be determined accurately.

If, on the other hand, the color saturation in the matching-target frameis low, i.e., if the matching-target frame is achromatic, the parameterα to a smaller value of, for instance, 0.2, so as to calculate thesimilarity factor by heavily weighting the brightness component, forevaluation over the chrominance components and ultimately improve thetracking performance.

When the matching-target frame is achromatic, and α is set to, forinstance, 0.2, the brightness component SAD_(Y) will be multiplied by0.8×2 and the chrominance components SAD_(Cr) and SAD_(Cb) will each bemultiplied by 0.2 in expression (12) as indicated in FIG. 5C. As aresult, the similarity factor SAD can be calculated by lowering theextent to which the chrominance components SAD_(Cr) and SAD_(Cb) factorinto the calculation. The similarity factor SAD thus calculated, lessaffected by SAD_(Cr) and SAD_(Cb), takes on the smallest value at anextraction position 5 b, i.e., the correct subject position, as shown inFIG. 5D, to allow the subject position to be determined accurately.

In the second embodiment described above, the control device 104calculates the similarity factor (SAD) by first normalizing theindividual component SAD factors and then weighting the normalizedbrightness component similarity factor (SAD_(Y)) or the normalizedchrominance component similarity factors (SAD_(cr) and SAD_(Cb)) basedupon the color saturation of the target frame. Through these measures,an advantage is achieved in that better subject tracking performance isassured.

—Variations—

It is to be noted that the subject tracking devices achieved in theembodiments described above allow for the following variations.

(1) In both the first embodiment and the second embodiment describedabove, the control device 104 calculates the similarity factor throughthe SAD (sum of absolute differences) method among various similarityfactor calculation methods. However, the present invention is notlimited to this example and it may be adopted equally effectively inconjunction with a similarity factor calculated through a method otherthan the SAD method, such as a similarity factor calculated through anSSD (sum of squared differences) method or by using a color histogram.

(2) The control device 104 in both the first embodiment and the secondembodiment executes subject tracking for live view images. However, thepresent invention is not limited to this example and provided that thecamera 100 is equipped with a dynamic image photographing function, thecontrol device 104 may execute frame-to-frame subject tracking for theframes of images constituting a dynamic image or moving image that hasalready been photographed, instead of live view images.

(3) In the first and second embodiments described above, the controldevice 104 in the camera 100 functions as a subject tracking device thatexecutes subject tracking through the processing executed as shown inFIG. 2. However, a program based upon which the processing in FIG. 2 isexecuted may be recorded in another terminal such as a personal computerso as to enable the terminal to execute the processing. In such a case,processing may be executed on dynamic image data expressing a dynamicimage photographed in the camera and taken into the terminal so as toexecute frame-to-frame subject tracking for the frames of imagesconstituting the dynamic image. In addition, the subject tracking deviceaccording to the present invention may be installed in a different typeof electronic device such as a camera-equipped portable telephone.

(4) In the first and second embodiments described above, similarityfactors are calculated in correspondence to a plurality of types ofcomponents, i.e., the brightness component (Y component) and thechrominance components (Cr component and Cb component), which assumespecific characteristic quantity values to characterize each image.However, similarity factors may be calculated in correspondence tocharacteristics components other than these. For instance, the Rcomponent, the G component and the B component may be assigned as theplurality of types of characteristics components expressing the inputimage. In such a case, similarity factors will be individuallycalculated in correspondence to the R component, the G component or theB component.

As long as the functions characterizing the present invention remainintact, the invention is in no way limited to the structural detailsdescribed in reference to the is embodiments. In addition, either of theembodiments may be adopted in combination with a plurality ofvariations.

Through either of the embodiments of the present invention describedabove, a similarity factor can be calculated accurately throughnormalization even when different gains are applied to various imagecomponents.

What is claimed is:
 1. A subject tracking device, comprising: a controldevice, including at least a CPU, the control device being configuredto: compare an input image assuming characteristics quantitiescorresponding to a plurality of characteristics components, with atemplate image assuming characteristics quantities corresponding to theplurality of characteristics components, and calculate a firstsimilarity factor indicating a level of similarity between the inputimage and the template image in correspondence to each of the pluralityof characteristics components; normalize the first similarity factorscorresponding to the plurality of characteristics components bymultiplying the first similarity factors by normalizing values used toequalize the first similarity factors corresponding to the individualcharacteristic components; and calculate a second similarity factorindicating a level of similarity between the input image and thetemplate image based upon results of normalization of the firstsimilarity factors.
 2. A subject tracking device according to claim 1,wherein the control device is further configured to: weight the firstsimilarity factors, having been calculated for the plurality ofcharacteristics components, in correspondence to characteristics of theinput image.
 3. A subject tracking device according to claim 1, wherein:the normalizing values are calculated and recorded into a storage mediumin advance; and the control device reads out the normalizing values fromthe storage medium and executes normalization.
 4. A subject trackingdevice according to claim 1, wherein: the plurality of characteristicscomponents are a brightness component and a chrominance component of theimage; and the control device calculates the first similarity factorindicating a level of similarity between the brightness component of theinput image and the brightness component of the template image and thefirst similarity factor indicating a level of similarity between thechrominance component in the input image and the chrominance componentin the template image.
 5. A camera equipped with a subject trackingdevice according to claim 1.